Binaural Rendering of Spherical Microphone Array Signals

The presentation of extended reality for consumer and professional applications requires major advancements in the capture and reproduction of its auditory component to provide a plausible listening experience. A spatial representation of the acoustic environment needs to be considered to allow for movement within or an interaction with the augmented or virtual reality. This thesis focuses on the application of capturing a real-world acoustic environment by means of a spherical microphone array with the subsequent head-tracked binaural reproduction to a single listener via headphones. The introduction establishes the fundamental concepts and relevant terminology for non-experts of the field. Furthermore, the specific challenges of the method due to spatial oversampling the sound field as well as physical limitations and imperfections of the microphone array are presented to the reader. The first objective of this thesis was to develop a software in the Python programming language, which is capable of performing all required computations for the acoustic rendering of the captured signals in real-time. The implemented processing pipeline was made publicly available under an open-source license. Secondly, specific parameters of the microphone array hardware as well as the rendering software that are required for a perceptually high reproduction quality have been identified and investigated by means of multiple user studies. Lastly, the results provide insights into how unwanted additive noise components in the captured microphone signals from different spherical array configurations contribute to the reproduced ear signals

End-to-End Magnitude Least Squares Binaural Rendering of Spherical Microphone Array Signals

Spherica1 microphone array (SMA) recordings are particularly suited for dynamic binaural rendering as the microphone signals can be decomposed into a spherical harmonic (SH) representation that can be freely rotated to match the head orientation of the listener. The rendering of such SMA recordings is a non-trivial task as the SH signals are impaired due to truncation of the SH decomposition order, spatial aliasing and the gain limitation of the employed radial filters. The perceptually most relevant consequence of this is an alteration of the magnitude transfer function at high frequencies. Previously, the magnitude least squares (MagLS) renderer for binaural rendering of SH signals was proposed to mitigate these effects under the assumption of ideal order-truncated plane waves, i.e., disregarding the influence of spatial aliasing as well as of non-ideal radial filters. Based on the MagLS renderer, we present a binaural rendering method for SMA recordings that integrates a comprehensive SMA model into the magnitude least squares objective. We evaluate the proposed end-to-end renderer by analyzing the reproduced binaural magnitude response. Our results suggest that the method significantly improves the high-frequency rendering mainly due to the inherent binaural diffuse-field equalization, while it achieves a slight improvement in the low and mid frequency range, where the error of the conventional method is already small. A reference implementation of the method accompanies this paper

End-to-End Magnitude Least Squares Binaural Rendering for Equatorial Microphone Arrays

We recently presented an end-to-end magnitude least squares (eMagLS) binaural rendering method for spherical microphone array (SMA) signals that integrates a comprehensive array model into a magnitude least squares objective to minimize reproduction errors. The introduced signal model addresses impairments due to practical limitations of spherical harmonic (SH) domain rendering, namely, spatial aliasing, truncation of the SH decomposition order, and regularized radial filtering. In this work, we improve the processing model when applied to the recently proposed equatorial microphone array (EMA) to facilitate three degrees-of-freedom head rotations during the rendering. EMAs provide similar accuracy to SMAs for sound fields from sources inside the horizontal plane while requiring a much lower number of microphones. We compare the proposed end-to-end renderers for both array types against a given binaural reference magnitude response. In addition to anechoic array simulations, the evaluation includes measured array room impulse responses to show the method’s effectiveness in minimizing high-frequency magnitude errors for all head orientations from SMAs and EMAs under practical room conditions. The published reference implementation of the method has been refined and now includes the solution for EMAs

Real-Time Implementation of Binaural Rendering of High-Order Spherical Microphone Array Signals

We present ReTiSAR (Real-Time Spherical Array Renderer), an open-source implementation of real-time binaural rendering of signals obtained from spherical microphone arrays. The implementation was performed in Python and bases on the previously published SOFiA toolbox as well as on sound_field_analysis-py. We can confirm that Python together with the other tools employed constitutes a viable framework for this kind of heavy-computation application even under real-time constraints. The current version of ReTiSAR is able to render signals of up to 8th order on a standard laptop computer

Spherical Harmonic Decomposition of a Sound Field Based on Microphones Around the Circumference of a Human Head

We present a method for decomposing a sound field into spherical harmonics (SH) based on observations of the sound field around the circumference of a human head. The method is based on the analytical solution for observations of the sound field along the equator of a rigid sphere that we presented recently. The present method incorporates a calibration stage in which the microphone signals for sound sources at a suitable set of calibration positions are projected onto the SH decomposition of the same sound field on the surface of a notional rigid sphere by means of a linear filtering operation. The filter coefficients are computed from the calibration data via a least-squares fit. We present an evaluation of the method based on binaural rendering of numerically simulated signals for an array of 18 microphones providing 8th SH order to demonstrate its effectiveness

A Head-Mounted Microphone Array for Binaural Rendering

We recently presented a method for obtaining a spherical harmonic representation of a sound field based on microphones along the equator of a rigid spherical object that ideally has a size similar to that of a human head. We refer to this setup as an equatorial microphone array. Even more recently, we presented an extension of this method that allows for employing a scattering object that is approximately spherical such as a human head. The present paper provides an overview as well as a juxtaposition of the two solutions. We present an instrumental evaluation based on the application of binaural rendering of the captured sound fields by analysing simulated binaural transfer functions of both methods for a variety of scenarios

The Far-Field Equatorial Array for Binaural Rendering

We present a method for obtaining a spherical harmonic representation of a sound field based on a microphone array along the equator of a rigid spherical scatterer. The two-dimensional plane wave de-composition of the incoming sound field is computed from the microphone signals. The influence of the scatterer is removed under the assumption of distant sound sources, and the result is converted to a spherical harmonic (SH) representation, which in turn can be rendered binaurally. The approach requires an order of magnitude fewer microphones compared to conventional spherical arrays that operate at the same SH order at the expense of not being able to accurately represent non-horizontally-propagating sound fields. Although the scattering removal is not perfect at high frequencies at low harmonic orders, numerical evaluation demonstrates the effectiveness of the approach

Perceptual Evaluation of Mitigation Approaches of Impairments due to Spatial Undersampling in Binaural Rendering of Spherical Microphone Array Data

Spherical microphone arrays (SMAs) are widely used to capture spatial sound fields that can then be rendered in various ways as a virtual acoustic environment (VAE) including headphone-based binaural synthesis. Several practical limitations have a significant impact on the fidelity of the rendered VAE. The finite number of microphones of SMAs leads to spatial undersampling of the captured sound field, which, on the one hand, induces spatial aliasing artifacts and, on the other hand, limits the order of the spherical harmonics (SH) representation. Several approaches have been presented in the literature that aim to mitigate the perceptual impairments due to these limitations. In this article, we present a listening experiment evaluating the perceptual improvements of binaural rendering of undersampled SMA data that can be achieved using state-of-the-art mitigation approaches. In particular, we examined the Magnitude Least-Squares algorithm, the Bandwidth Extraction Algorithm for Microphone Arrays, Spherical Head Filters, SH Tapering, and a newly proposed equalization filter. In the experiment, subjects rated the perceived differences between a dummy head and the corresponding SMA auralization. We found that most mitigation approaches lead to significant perceptual improvements, even though audible differences to the reference remain

Spherical Harmonic Decomposition of a Sound Field Using Microphones on a Circumferential Contour Around a Non-Spherical Baffle

Spherical harmonic\ua0(SH) representations of sound fields are usually obtained from microphone arrays with rigid spherical baffles whereby the microphones are distributed over the entire surface of the baffle. We present a method that overcomes the requirement for the baffle to be spherical. Furthermore, the microphones can be placed along a circumferential contour around the baffle. This greatly reduces the required number of microphones for a given spatial resolution compared to conventional spherical arrays. Our method is based on the analytical solution for SH\ua0decomposition based on observations along the equator of a rigid sphere that we presented recently. It incorporates a calibration stage in which the microphone signals due to a suitable set of calibration sound fields are projected onto the SH\ua0decomposition of those same sound fields on the surface of a notional rigid sphere by means of a linear filtering operation. The filter coefficients are computed from the calibration data via a least/squares fit. We present an evaluation of the method based on the application of binaural rendering of the SH\ua0decomposition of the signals from an 18/element array that uses a human head as the baffle and that provides 8th ambisonic order. We analyse the accuracy and robustness of our method based on simulated data as well as based on measured data from a prototype

ISS emergency scenarios and a virtual training simulator for Flight Controllers

The current emergency response concept for the International Space Station (ISS) includes the support of the Flight Control Team. Therefore, the team members need to be trained in emergencies and the corresponding crew procedures to ensure a smooth collaboration between crew and ground. In the case where the astronaut and ground personnel training is not collocated it is a challenging endeavor to ensure and maintain proper knowledge and skills for the Flight Control Team. Therefore, a virtual 3D simulator at the Columbus Control Center (Col-CC) is presented, which is used for ground personnel training in the on-board emergency response. The paper briefly introduces the main ISS emergency scenarios and the corresponding response strategy, details the resulting learning objectives for the Flight Controllers and elaborates on the new simulation method, which will be used in the future. The status of the 3D simulator, first experiences and further plans are discussed